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Section 10 

Multivariate normal distributions and 
CLT. 

Let P be a probability distribution on M*^ and let 

g{t) = J e*(*'^)dP(a;). 
We proved that P"^ = P * J\f{0, a^J) has density 

p^ix) = (27r)-^y"g(t)e-'(*'^)-5'^'l*l'dt. 

Lemma 20 (Fourier inversion formula) If J \g{t)\dt < oo then P has density 

p{x) ^ (27r)-'=y" 5(t)e-*(*'^)rfi. 

Proof. Since 
pointwise as a ^ and 

g(^fy-^{t,x)-la^\t\^ < \g{t)\ - integrablc, 
by dominated convergence theorem p'^(x) p{x). Since P"^ ^ P weakly, for any / G Cb(M''), 

J f{x)p''{x)dx ^ j f{x)dF{x). 

On the other hand, since 

\p^x)\ < (277)-'= y \g{t)\dt<^, 
by dominated convergence theorem, for any compactly supported f G Ch 



Therefore, for any such /, 



f{x)p'^{x)dx j f{x)p{x)dx. 
j f{x)dF{x) = J f{x)p{x)dx. 
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It is now a simple exercise to show that for any bounded open set U, 

/ dF{x) = / p{x)dx. 
Ju Ju 

This means that P restricted to bounded sets has density p{x) and, hence, on entire R'^. 

□ 

For a random vector X = {Xi, . . . ,Xk) gR'' we denote EX = {EXi, . . . ,EXfc). 

Theorem 24 Consider a sequence {Xi)i>i of i.i.d. random vectors on M'^ such that EXi = 0,E\Xi\'^ < oo. 
Then c(^-^^ converges weakly to distribution P which has characteristic function 

= e" 5 where C = Cov{Xi) = (EXi^iXij) . (10.0.1) 

Proof. Consider any t G M'=. Then Zi = {t, Xi) is i.i.d. real- valued sequence and by the proof of the CLT on 
the real line, 

]Ee'(«.^) = Egi;^ T.i(t,x,) _^ g-iVar((t,xO) = ^-h(Ct,t) 

as n — > oo, since 

Var(iiXi,i + • • • + tkXi^k) =J2titjn,iXi,j = {Ct,t) = t^Ct. 
The sequence is uniformly tight on M*^ since 

It remains to apply Lemma 19 from previous section. 

□ 

The covariance matrix C = Cov(X) is symmetric and non-negative definite, {Ct,t) — E{t,Xy > 0. 

The unique distribution with c.f. in (10.0.1) is called a multivariate normal distribution with covariance 
C and is denoted by ^^(0, C) . It can also be defined more constructively as follows. Consider an i.i.d. sequence 
gi,...,gn oiM{Q, 1) r.v. and lei g = {gi, . . . ,gn)^ ■ Given a,kxn matrix the covariance matrix of Ag e M.'^ is 

C = CoY{Ag) = EAg{Agf = AEgg'^A'^ = AA^ . 

The c.f. of Ag is 

This means that Ag ^ A/'(0,C). Interestingly, the distribution of Ag depends only on AA^ and does not 
depend on the choice of n and A. 

Exercise. Show constructively, using linear algebra, that the distribution of Ag and Bg' is the same if 
AA^ = BB^. 

□ 

On the other hand, given a covariance matrix C one can always find A such that C = AA^ . For 
example, let C = QDQ^ be its eigenvalue decomposition for orthogonal matrix Q and diagonal D. Since 
C is nonncgativc definite, the elements of D are nonnegative. Then, one can take n = k and A = C^l"^ := 
QD^/^QT orA = QD^/\ 

Density in the invertible case. Suppose det(C) ^ 0. Take A such that C = AA"^ so that Ag ~ 
j\/'(0, C). Since the density of g is 



i<k ^ ^ 
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for any set f2 C M*' we can write 

¥{AgGn)=¥{gGA-^n)= [ (^Yexp(~l\x\^)dx. 
Let us now make the change of variables y = Ax ov x = A~^y. Then 

But since 

det(C) = det(AA^) = det(A) det(A^) = det{Af 



we have | det(>l)| = ^/dct{C'). Also 

\A-'y\^ = {A-^yfiA-'y) = y^{A^)-'A-'y = y^{AA^)-'y = y^C-'y. 

Therefore, we get 

This means that the distribution A/'(0, C) has the density 

/ 1 1 



27r/ v'det(C) 



General case. Let us take, for example, a vector X = QD^/^g for i.i.d. standard normal vector g so 
that X ~ .A/'(0, C).li q\, . . . ,qk are the column vectors of Q then 

X = QD^/^g = {X\'^gr)qi + ■■■ + {\T9n)qk- 

Therefore, in the orthonormal coordinate basis qi, . . . ,qk sl random vector X has coordinates A^^^gi , . . . , A^^^g^. 
These coordinates are independent with normal distributions with variances Ai,...,Aft correspondingly. 
When det(C) = 0, i.e. C is not invertible, some of its eigenvalues will be zero, say, A„+i = . . . = Afc = 0. 
Then the random X vector will be concentrated on the subspace spanned by vectors gi , . . . , g„ but it will 
not have density on the entire space M*^ . On the subspace spanned by vectors gi , . . . , g„ a vector X will have 
a density 
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Let us look at a couple of properties of normal distributions. 
Lemma 21 // X ~ Af{0, C) on K*^ and A : K'^ ^ is linear then AX ~ jV(0, ACA^) on 
Proof. The c.f. of AX is 



Lemma 22 X is normal on M*^ iff {t, X) is normal on R for all t £R'^. 
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Proof. " =>" . The c.f. of real- valued random variable {t, X) is 

which means that {t,X) ~ J\f{0, {Ct,t)). 
"<^". If {t,X) is normal then 

]Egi(t,^) = g-|(ct,t) 

because the variance of {t,X) is {Ct,t). 

a 

Lemma 23 iyci Z = {X, Y) where X = {Xi, . . . , X^) and Y = (Yi, . . . , Yj) and suppose that Z is normal 
on W~^^ . Then X and Y are independent iff Coy{Xm, Yn) = for all m, n. 
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Proof. One way is obvious. The other way around, suppose that 

C = Cov(Z) 
Then the c.f. of Z is 

]Egi(t,-z) ^ g-|(ct,t) ^ g-i(Dti,ti)-i(Ft2,t2) = Ee'**^'^^Ee'^*^''^\ 
where t = {ti,t2)- By uniqueness, X and Y are independent. 

□ 

Lemma 24 ( Continuous Mapping.) Suppose that P„ — > P on X and G : X ^ Y is a continuous map. Then 
P„ o — > P o on Y. In other words, if r.v. Zn ^ Z weakly then G{Zn) G{Z) weakly. 

Proof. This is obvious, because for any / e Cii{Y), we have f oG G Cb{X) and therefore, 

E/(G(Z„))^E/(G(Z)). 



Lemma 25 7/P„ ^ P on M'^ and Q„ ^ Q on then P„ x Q„ ^ P x Q on 
Proof. By Fubini theorem. The c.f. 

By Lemma 19 it remains to show that (P„ x Q„) is uniformly tight. By Theorem 21, since P„ — > P, (P„) is 
uniformly tight. Therefore, there exists a compact K on M*^ such that P„(ii') > 1 — e. Similarly, for some 
compact K' on M™, Q„(if') > 1 - e. We have, 

P„ X Qn{K xK')>l-2e 

and K X K' ]s & compact on 

□ 

Corollary 1 //P„ ^ P and Q„ ^ Q both on then P„ * Q„ ^ P * Q. 

Proof. Since a function G : — > R*^ given by G{x, y) = x + y \s continuous, by continuous mapping 
lemma, 

Pn * Qn = (Pn X Q„) O G"^ ^ (P X Q) O = P * Q. 
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